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Program Description^ 

Accelerated Reader™ is a computerized supplementary reading pro- 
gram that provides guided reading instruction to students in grades 
K-12. It aims to improve students’ reading skills through reading 
practice and by providing frequent feedback on students’ progress 
to teachers. The Accelerated Reader™ program requires students 
to select and read a book based on their area of interest and read- 
ing level. Upon completion of a book, students take a computerized 
quiz based on the book’s content and vocabulary. Quiz performance 
allows teachers to monitor student progress and to identify students 
who may need additional reading assistance. This review of the 
program for the Beginning Reading topic area focuses on studies that 
examine outcomes for beginning readers in grades K-3. 

Research^ 

The What Works Clearinghouse (WWC) identified two studies of 
Accelerated Reader™ that both fall within the scope of the Beginning 
Reading topic area and meet WWC group design standards. Both 
studies meet WWC group design standards without reservations. 
Together, these studies included 265 beginning readers in grades 1-3 
in four schools. 
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This intervention report presents findings 
from a systematic review of Accelerated 
Reader™, conducted using the WWC 
Procedures and Standards Handbook, 
version 3.0, and the Beginning Reading 
review protocol, version 3.0. 


The WWC considers the extent of evidence tor Accelerated Reader™ 
on the reading achievement of beginning readers to be small for two 
student outcome domains— comprehension and reading fluency. 

There were no studies that met WWC group design standards in two other domains specified by the Beginning 
Reading topic area, so this intervention report does not report on the effectiveness ot Accelerated Reader™ for 
those domains. (See the Effectiveness Summary on p. 4 for more details of effectiveness by domain.) 


Effectiveness 

Accelerated Reader™ was found to have mixed effects on comprehension and no discernible effects on reading 
fluency for beginning readers. 


Table 1. Summary of findings^ 




Improvement index (percentile points) 




Outcome domain 

Rating of effectiveness 

Average 

Range 

Number of 
studies 

Number of 
students 

Extent of 
evidence 

Comprehension 

Mixed effects 

-2 

-11 to +7 

2 

265 

Small 

Reading fluency 

No discernible effects 

3 

na 

1 

32 

Small 


na = not applicable 
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Program Information 

Background 

The /Acce/erafed Reader™ software prototype was created in 1984 by Judi Paul, and was first marketed to schools 
in 1986. In 1992, a series of studies on the program were started to identify best practices related Xo Accelerated 
Reader™. These efforts led to the development of the Accelerated Reader™ Best Classroom Practices, first intro- 
duced to educators in 1996 through professional development seminars. For several years. Renaissance Learning 
sold Accelerated Reader™ Best Classroom Practices together with Accelerated Reader™ under the name Read- 
ing Renaissance. In 2004, Renaissance Learning introduced a hosted, web-based version ot Accelerated Reader™ 
featuring a subscription service payment model, called Renaissance Place™. In 2006, this was replaced hy Accel- 
erated Reader Enterprise™, which was hosted on the same web-based platform and included access to the entire 
library oX Accelerated Reader™ quizzes, in 2014, Renaissance Learning released a new version, caWed Accelerated 
Reader 360™, which expanded the Independent reading practice component to include (1) both fiction and nonfic- 
tion literature and (2) a teacher dashboard that summarizes results for each student and across students. The WWC 
refers to all of these packages as Accelerated Reader™ In this intervention report. This program is distributed by 
Renaissance Learning, Inc. Address: PO Box 8036, Wisconsin Rapids, Wl 54495-8036. Email: answers@renais- 
sance.com. Web: http://www.renaissance.com/. Telephone: (800) 338-4204. 

Program details 

As a first step, students independently use Accelerated Reader™, a computerized supplementary reading pro- 
gram, to determine their reading level by completing a norm-referenced, standardized measure of general reading 
achievement. The teacher and student then use these reading level data to develop personalized reading practice 
goals along three dimensions: comprehension, quantity, and complexity. The students develop their reading skills 
by participating in reading practice sessions. In the beginning of a session, students choose to read books in 
their area of interest and within a recommended readability range from a list of books available from Accelerated 
Reader™. Depending on their age and skill level, students read the books with the help of a teacher— for example, 
the teacher may read books to students or with students using a paired-reading technique— or independently. 
Students reading with the help of a teacher transition to independent reading as they develop decoding skills. 

After reading each book, students take a comprehension quiz and earn points based on the number of correct 
responses, the length of the book, and the readability level of the book. Teachers can use data from the quizzes to 
monitor student progress, adjust students’ reading ranges, or identify students who may need additional reading 
assistance. Teachers can use points to set goals for the quantity and quality of reading practice for each student. 
Point accumulation, as well as other optional teacher-provided rewards, are intended to motivate student learning; 
however, the intervention does not recommend or require the use of external rewards. The developer recommends 
using Accelerated Reader™ during In-class time dedicated to reading practice for at least 35 minutes. 


Cost 

Accelerated Reader™ and Accelerated Reader 360™ are sold on an annual, per-student subscription. Additional 
cost information is available from the distributor. 
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Research Summary 

The WWC identified 1 1 eligible studies that investigated the effects of 
Accelerated Reader™ on the reading skills of beginning readers. An 
additionai 60 studies were identified but do not meet WWC eligibiiity 
criteria for review in this topic area. Citations for ail 71 studies are in the 
References section, which begins on p. 6. 

The WWC reviewed the 1 1 eligibie studies against group design stan- 
dards. Two studies are randomized controlied trials that meet WWC group design standards without reservations. 
Those two studies are summarized in this report. Nine studies do not meet WWC group design standards. 

Summary of studies meeting WWC group design standards without reservations 

Bullock (2005) conducted a cluster randomized controlied trial that examined the effect ot Accelerated Reader™ 
on the reading achievement of students In one elementary school near Eugene, Oregon. Within each of two class- 
rooms, students were ordered by baseline oral reading fluency score. Students whose rank order was an odd 
number were placed into one group, and students whose rank order was an even number were placed into another 
group. Within each classroom, one group of students was then randomly assigned to the intervention condition, 
while the other group was assigned to the comparison condition. The WWC based its effectiveness rating on find- 
ings from 32 third-grade students from two classrooms; 16 students were assigned to the intervention group and 
1 6 to the comparison group. Students in the intervention group used Accelerated Reader™ for 1 0 weeks, spending 
at least 90 minutes a week independently reading trade books and taking Accelerated Reader™ quizzes on each 
book. The comparison group also spent at least 90 minutes per week reading independently, and had the option to 
choose any book available in the school library, but did not use Accelerated Reader™ software. Each week, both 
classrooms also visited a library for 30 minutes. Students were tested immediately prior to and after the 1 0-week 
intervention period. Outcomes for an additional 82 students from two fourth-grade and two fifth-grade classrooms 
were not used to determine the WWC effectiveness rating because these grades are out of the scope of the Begin- 
ning Reading protocol. The study did not specify which version of the Accelerated Reader™ software was used. 

Shannon, Styers, and Siceloff (2010) conducted a cluster randomized controlled trial that examined the effect of 
Accelerated Reader™ on the reading achievement of students in three Catholic schools in a large city in the North 
Central region in the United States. Within each grade and school block, evaluators randomly assigned participating 
classrooms to intervention or comparison conditions. The WWC based its effectiveness rating on findings from 
233 students in grades 1-3 from 14 classrooms; 144 students in eight classrooms were assigned to the intervention 
group, and 89 students in six classrooms were assigned to the comparison group. Over the course of the school year, 
students in the intervention condition used Accelerated Reader™ for 4-5 days a week, spending 30-45 minutes per 
day practicing independent reading and taking quizzes on each book. Comparison group teachers reported using 
30 minutes or fewer of daily independent reading in their classrooms. Students were tested in the beginning of 
the year, mid-year, and at the end of the school year. The study did not specify which version of the Accelerated 
Reader™ software was used. 

Summary of studies meeting WWC group design standards with reservations 

No studies of Accelerated Reader™ met WWC group design standards with reservations. 


Table 2. Scope of reviewed research 


Grades 

1-3 

Delivery method 

Individual 

Program type 

Supplement 
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Effectiveness Summary 

The WWC review of studies for the Beginning Reading topic area includes outcomes for students in four domains: 
alphabetics, comprehension, reading fluency, and general reading achievement. The two studies of Accelerated 
Reader™ that meet WWC group design standards reported findings in two of the four domains: (a) comprehension 
and (b) reading fluency. The findings below present the authors’ estimates and WWC-calculated estimates of the 
size and statistical significance of the effects of Accelerated Reader™ on beginning readers. For a more detailed 
description of the rating of effectiveness and extent of evidence criteria, see the WWC Rating Criteria on p. 20. 

Summary of effectiveness for the comprehension domain 


Table 3. Rating of effectiveness and extent of evidence for the comprehension domain 


Rating of effectiveness 

Criteria met 

Mixed effects 

Evidence of Inconsistent effects. 

In the two studies that reported findings, the estimated impact of the intervention on outcomes in the comprehen- 
sion domain were (1) a statistically significant positive etfect and (2) a substantively important negative effect. 

Extent of evidence 

Criteria met 

Small 

Two studies that included 265 students in four schools reported evidence of effectiveness in the comprehen- 
sion domain. 


Two studies that meet WWC group design standards without reservations reported findings in the comprehen- 
sion domain. 

Bullock (2005) found, and the WWC confirmed, effects of Accelerated Reader™ that were negative but not sta- 
tistically significant, as measured by the STAR Reading Test. The effect size was large enough to be considered 
substantively important according to WWC criteria. The WWC characterizes this study finding as a substantively 
important negative effect. 

Shannon et al. (2010) found, and the WWC confirmed, effects of Accelerated Reader™ that were positive and sta- 
tistically significant, as measured by the STAR Reading Test. The WWC characterizes this study finding as a statisti- 
cally significant positive effect. 

Thus, for the comprehension domain, one study found a negative effect that was substantively important but not 
statistically significant, and one study found a positive and statistically significant effect. This results in a rating of 
mixed effects, with a small extent of evidence. 


Accelerated Reader™ Updated June 2016 


Page 4 




WWC Intervention Report 


Summary of effectiveness for the reading fluency domain 

Table 4. Rating of effectiveness and extent of evidence for the reading fluency domain 


Rating of effectiveness 

Criteria met 

No discernible effects 

No affirmative evidence of effects. 

In the one study that reported findings, the estimated impact of the intervention on outcomes In the reading fiu- 
ency domain was neither statistically significant nor large enough to be substantively important. 

Extent of evidence 

Criteria met 

Small 

One study that included 32 students in one school reported evidence of effectiveness in the reading ftocy domain. 


One study that meets WWC group design standards without reservations reported findings in the reading fluency 
domain. 

Bullock (2005) found, and the WWC confirmed, effects Accelerated Reader'^^ that were not statistically sig- 
nificant, based on the Oral Reading Fluency subtest of the Dynamic Indicators of Basic Early Literacy Skills 
(DIBELS). The effect size was not large enough to be considered substantively important according to WWC 
criteria. The WWC characterizes this study finding as an indeterminate effect. 

Thus, for the reading fluency domain, one study reported no statistically significant or substantively Important 
effects. This results in a rating of no discernible effects, with a small extent of evidence. 
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Solley, K. (2011). Accelerated Reader can be an effective tool to encourage and bolster student reading. Knowledge 
Quest, 39(4), 46-49. The study is ineligible for review because it does not use an eligible design. 

Steele, C. T. (2003). The effectiveness of the Accelerated Reader program on the reading level of second-grade 
students as measured by the student test for assessment of reading (Doctoral dissertation). Available from 
ProQuest Dissertations and Theses database. (UMI No. 3080207) The study Is Ineligible for review because 
It does not use an eligible design. 

Topping, K. J. (2006). Accelerated Reader in specialist schools. Scotland: Centre for Peer Learning, University of 
Dundee. The study is ineligible for review because it does not use an eligible design. 

Topping, K. J., & Paul, T. (1999). Computer-assisted assessment of practice at reading: A large scale survey using 
Accelerated Reader data. Reading & Writing Quarterly, 15(3), 213-231 . The study is ineligible for review 
because it does not use an eligible design. 

Topping, K. J., Samuels, J., & Paul, T. (2007). Computerized assessment of independent reading: Effects of imple- 
mentation quality on achievement gain. School Effectiveness and School Improvement, 18(2), 191-208. The 
study is ineligible for review because it does not use an eligible design. 

Topping, K. J., Samuels, J., & Paul, T. (2007). Does practice make perfect? Independent reading quantity, quality 
and student achievement. Learning and Instruction, 17(3), 253-264. The study is ineligible for review because 
it does not use an eligible design. 

Topping, K. J., & Sanders, W. L. (2000). Teacher effectiveness and computer assessment of reading: Relating value 
added and learning Information system data. School Effectiveness and School Improvement, 11(3), 305-337. 
The study is ineligible for review because it does not use an eligible design. 

Additional source: 

Renaissance Learning. (2000). Accelerated Reader and Reading Renaissance lead to increased teacher 
effectiveness. Wisconsin Rapids, Wl: Author. 

Townsend, K. (2007). Accelerated Reader: Qptimal conditions for reading achievement using a computer informa- 
tion system (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 
3267051) The study Is ineligible for review because it does not use an eligible design. 
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Walberg, H. (2001). Final evaluation of the reading initiative: Report to the J. A. & Kathryn Albertson Foundation 
Board of Directors. Wisconsin Rapids, Wl: Renaissance Learning. The study is ineiigible for review because 
it does not use an eligibie design. 

White, R., & Reisner, E. (2007). Model literacy programs: Save the Children: Evaluation findings from the 2005-06 
school year. Washington, DC: Poiicy Studies Associates. http://files.eric.ed.gov/fulitext/ED498796.pdf. The 
study is ineligibie for review because it does not use an eligibie design. 

Yee, V. N. (2007). An evaluation of the impact of a standards-based intervention on the academic achievement of 
English language learners (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. 
(UMI No. 3261828) The study is ineligible for review because it does not use a sample aligned with the protocol. 
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Appendix A.1: Research detaiis for Buiiock (2005) 

Bullock, J. C. (2005). Effects of the Accelerated Reader on reading performance of third, fourth, and 
fifth-grade students in one western Oregon elementary school (Doctoral dissertation). Available 
from ProQuest Dissertations and Theses database. (UMI No. 3181085) 


Tabie A1. Summary of findings Meets WWC group design standards without reservations 




Study findings 

Outcome domain 

Sample size 

Average improvement index 
(percentile points) 

Statistically significant 

Comprehension 

32 students 

-11 

No 

Reading fluency 

32 students 

+ 3 

No 


Setting The study was conducted in one elementary school in Lane County, Oregon. 

Study sample The study examined students in grades 3-5 in one elementary school; however, this WWC 
review focuses on results for the third-grade students only, as specified in the Beginning 
Reading protocol. This third-grade sample consisted of 32 students from two classrooms. 
Within each classroom, the students were ordered by baseline oral reading fluency score; stu- 
dents whose rank order was an odd number were placed Into one group, and students whose 
rank order was an even number were placed into another group. Within each classroom, one 
group of students was then randomly assigned to the intervention condition (Accelerated 
Reader™), while the other group was assigned to the comparison condition (business-as- 
usual). The Accelerated Reader™ group consisted of 1 1 female and five male students, while 
the business-as-usual comparison group consisted of nine female and seven male students. 
There was no attrition of students between the pretest and the posttest. 

At the time of the study, the participating elementary school had approximately 325 students. 
The school population was approximately 91 % White, 5% other race/ethnicIty, and 4% 
Hispanic. A majority (60.5%) of the overall school population were eligible for free or reduced- 
price lunch. 


IntcrVGntion students in the intervention group used Accelerated Reader™ over a 10-week period. They 
group were provided with a minimum of 90 minutes per week of independent reading during class 
time and required to visit a library for 30 minutes per week. They also had access to Acceler- 
ated Reader™ books that were within their reading level and took Accelerated Reader™ quiz- 
zes on each book. The Accelerated Reader™ intervention was used to supplement a standard 
core reading curriculum; however, the author did not identify which core curriculum was used. 


Comparison in addition to using the same core reading curriculum as the Intervention group, students 
group in the comparison group spent at least 90 minutes a week over a 10-week period reading 

independently and were required to visit a library for 30 minutes per week. However, they were 
free to choose any book available In the school library and did not use Accelerated Reader™ 
software. Comparison group students were asked to keep track of the books they read. 
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Outcomes and 
measurement 

Comprehension was measured using the STAR Reading Test, a norm-referenced, computer- 
adaptive test. Reading fiuency was measured using the Orai Reading Fiuency subtest of the 
Dynamic indicators of Basic Eariy Literacy Skiiis (DiBELS), which was administered individu- 
aiiy by a trained educationai assistant. The two assessments were administered twice: at 
the beginning and end of the 10-week study period. For a more detaiied description of these 
outcome measures, see Appendix B. 

Support for 
implementation 

No information is provided about any speciai training or support for impiementation offered to 
these teachers. Reading ciasses for the intervention and comparison groups were taught by 
the schooi’s reguiar teachers. 


Appendix A.2: Research detaiis for Shannon et ai. (2010) 

Shannon, L. C., Styers, M. K., &Siceloff, E. R. (2010). A final report for the evaluation of Renaissance 
Learning’s Accelerated Reader program. Charlottesville, VA: Magnolia Consulting. 

Tahie A2. Summary of findings Meets WWC group design standards without reservations 


study findings 
Average improvement index 

Outcome domain Sample size (percentile points) Statistically significant 


Outcome domain 

study findings 

Average improvement index 

Sample size (percentile points) Statistically significant 

Comprehension 

233 students +7 Yes 

Setting 

This study took piace in three Cathoiic private schoois in a iarge city in the North Centrai 
region of the United States. 

Study sample 

The study examined students in grades 1-4 in three schoois; however, this WWC review 
focuses on students in grades 1-3 oniy, as specified in the Beginning Reading protocoi. This 
sampie consisted of 233 students from 14 ciassrooms. Within each grade, teachers were 
randomiy assigned to either the intervention condition (Accelerated Reader™) or the compari- 
son condition (business-as-usuai). The study had no attrition at the ciassroom ievei and iow 
attrition at the student ievei based on WWC “iiberai” attrition assumptions. 

The fuii study sampie, which inciuded 344 first, second, third, and fourth-grade students in 19 
ciassrooms, was approximateiy eveniy spiit between femaie and maie students (52% vs. 48%, 
respectiveiy). The majority of study participants (66%) were White, 13% were Asian/Pacific 
isiander, and 1 1 % were Hispanic. None of the students were ciassified as Engiish iearners, 
iess than 2% were receiving speciai education, and iess than 3% were eiigibie for free or 
reduced-price iunch. 

Intervention 

group 

Students in the intervention group used Accelerated Reader™ over the course of 1 schooi 
year. Teachers reported using the intervention daiiy (on average, 4.9 out of 5 days a week) in 
their ciassrooms, aiiowing for 30-45 minutes of independent reading practice per day. During 
this time, students read books within their reading and interest ieveis and took Accelerated 
Reader™ quizzes that tested their understanding of the book. The intervention was used to 
suppiement the standard reading curricuium (Scott Foresman). 
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Comparison 

group 


Outcomes and 
measurement 


Support for 
implementation 


Students in the comparison group were taught using a similar set of reading curricula as the 
intervention group, but without the addition of Accelerated Reader™. Both groups supple- 
mented their core curricula— Scoff Foresman— with other materials. 

Comprehension was measured using the STAR Reading Test, a norm-referenced, computer- 
adaptive test. The assessment was administered three times: at the beginning, middle, and 
end of the school year. This report focuses on impacts only at the end of the year, which coin- 
cided with the end of the intervention. 

In addition to the findings for the combined sample of students in grades 1-3, the study 
presented supplemental findings for grade-specific subgroups. The findings for students in 
first grade on the STAR Reading Test did not meet WWC design standards because attrition 
rates were high, and baseline equivalence of the Intervention and comparison groups’ ana- 
lytic sample was not demonstrated. The findings for students in grades 2 and 3 on the STAR 
Reading Test are presented in Appendix D. These supplemental findings do not factor into the 
intervention’s rating of effectiveness. For a more detailed description of these outcome mea- 
sures, see Appendix B. 

Renaissance Learning, the program developer, provided a training seminar to intervention 
teachers at the beginning of the school year and conducted training visits to each school 
throughout the study period. 
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Appendix B: Outcome measures for each domain 


Comprehension 


Comprehension 


STAR Reading Test 

This is a norm-referenced computer-adaptive test which assesses a student’s vocabuiary-in-context knowiedge 
and reading comprehension (as oited in Buiiock, 2005 and Shannon et ai., 2010). Students in grades K-2 
answer 25 vooabuiary-in-context items, which require a student to tiii in a missing word by gathering contextuai 
ciues from the rest of the sentence, in grades 3 and higher, the test consists of two parts: (1) 20 vooabuiary- 
in-context items, and (2) live passages toiiowed by muitipie-choioe comprehension questions. This assessment 
takes 10-15 minutes to compiete. The test was deveioped by Renaissance Learning, the deveioper of Acceler- 
ated Reader'^^. For grades 1-5, spiit-haif reiiabiiity of the STAR Reading Test ranges from .89 to .90, whiie the 
test-retest reiiabiiity ranges from .84 to .91 (Shannon et ai., 2010). 

Reading fluency 


Oral Reading Fluency subtest of the 
Dynamic Indicators of Basic Early 
Literacy Skills (DIBELS) 

This is a standardized assessment of a student’s reading accuracy and fiuency (as cited in Buiiock, 2005). it 
was administered individuaiiy by a trained educationai assistant. This assessment measures the number of 
words read correctiy in 1 minute. 


Reading fluency 
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Appendix C.1: Findings inciuded in the rating for the comprehension domain 


Mean 

(standard deviation) WWC calculations 

Study Sample Intervention Comparison Mean Effect Improvement 
Outcome measure sample size group group difference size index p-value 


Bullock, 2005^ 

STAR Reading Test Grades 4 groups/ 412.40 462.30 -49.90 -0.29 -11 nr 

32 students (149.50) (182.40) 

Domain average for comprehension (Bullock, 2005) -0.29 -11 Not 

statistically 

significant 

Shannon etal., 2010'’ 

STAR Reading Test Grades 14 classrooms/ 426.13 396.30 29.83 0.18 +7 .04 

1-3 233 students (159.92) (166.08) 

Domain average for comprehension (Shannon et al., 2010) 0.18 +7 Statistically 

significant 


Domain average for comprehension across all studies -0.05 -2 na 


Table Notes: For mean difference, effecf size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for aii individuais who 
are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, refiecting the change 
in an average individuai’s percentiie rank that can be expected if the individuai is given the intervention. The WWC-computed average effect size is a simpie average rounded to 
two decimai piaces; the average improvement index is caicuiated from the average effect size. The statisticai significance of each study's domain average was determined by the 
WWC. Some statistics may not sum as expected due to rounding, na = not appiicabie. nr = not reported. 

For Buiiock (2005), a correction for ciustering was needed and resuited in a WWC-computed p-vaiue of .62 for STAR Reading Test; therefore, the WWC does not find the resuit to be 
statisticaiiy significant. The effect size was caicuiated by using unadjusted posttest means and standard deviations (reported on p. 30 of the study). Piease see the WWC Procedures 
and Standards Flandbook (version 3.0) for more information. This study is characterized as having a substantiveiy important negative effect because the effect is negative and not 
statisticaiiy significant but is substantiveiy important. For more information, piease refer to the WWC Standards and Procedures Flandbook (version 3.0), p. 26. Note that a siightiy 
different improvement index (of -0.12 and corresponding effect size of -0.30 that did not use Fledges’ small sample size correction in the denominator of effect size formula) was 
reported in the previous intervention report (2008) for the same outcome analysis. 

For Shannon et al. (201 0), the WWC did not need to make corrections for clustering, multiple comparisons, or to adjust for baseline differences. The p-value presented here was 
reported in the original study. The intervention group mean was calculated by adding the hierarchical linear modeling (HLM) coefficient to the comparison group posttest mean. Post- 
test standard deviations and the comparison group mean were calculated by aggregating corresponding statistics across the three grades of students (reported on p. 56 of the study). 
This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more information, please refer to 
the WWC Standards and Procedures Flandbook (version 3.0), p. 26. 
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Appendix C.2: Findings inciuded in the rating for the reading fluency domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index p-value 

Bullock, 2005^ 

Oral Reading Fluency subtest 
of the Dynamic Indicators 

Grade 3 

4 groups/ 

32 students 

116.30 

(40.90) 

112.80 

(55.40) 

3.50 

0.07 

+3 nr 


of Basic Early Literacy Skills 
(DIBELS) 

Domain average for reading fluency (Bullock, 2005) 0.07 +3 Not 

statistically 

significant 


Domain average for reading fluency across all studies 0.07 +3 na 


Table Notes: For mean difference, effecf size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for aii individuais who are 
given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, rejecting the change in an 
average individuai's percentiie rank that can be expected if the individuai is given the intervention, na = not appiicabie. nr = not reported. 

For Buiiock (2005), a correction for ciustering was needed and resuited in a WWC-computed p-vaiue of .90 for DiBELS; therefore, the WWC does not find the resuit to be statisti- 
caiiy significant. The effect size was caicuiated by using unadjusted posttest means and standard deviations (reported on p. 30 of the study). Piease see the WWC Procedures and 
Standards Handbook (version 3.0) for more information. This study is characterized as having an indeterminate effect because the estimated effect is neither statisticaiiy significant 
nor substantiveiy important. For more information, piease refer to the WWC Standards and Procedures Handbook (version 3.0), p. 26. 
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Appendix D: Description of suppiementai findings for the comprehension domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Shannon etal., 2010^ 

STAR Reading Test 

Grade 2 

4 Classrooms/ 
69 students 

453.14 

(86.06) 

398.30 

(128,68) 

54.84 

0.50 

-f19 

nr 

STAR Reading Test 

Grade 3 

5 classrooms/ 
82 students 

510,08 

(127.08) 

475.65 

(185.04) 

34,43 

0.23 

-f9 

nr 


Table Notes: The supplemental findings presented in this tabie are additionai findings from studies in this report that meet WWC design standards without reservations, but do not 
factor into the determination of the intervention rating. For mean difference, effect size, and improvement index vaiues reported in the tabie, a positive number favors the interven- 
tion group and a negative number favors the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average 
change expected for aii individuais who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation 
of the effect size, rejecting the change in an average individuai's percentiie rank that can be expected if the individuai is given the intervention. Some statistics may not sum as 
expected due to rounding, nr = not reported. 

For Shannon et ai. (2010), a correction for ciustering was needed for STAR Reading and resuited in a WWC-computed p-vaiue of .34 for grade 2 students and .64 for grade 3 students; 
therefore, the WWC does not find these resuits to be statisticaiiy significant. The WWC caicuiated the program group mean using a difference-in-differences approach by adding the 
impact of the program (i.e., difference in mean gains between the intervention and comparison groups) to the unadjusted comparison group posttest means. Piease see the WWC 
Procedures and Standards Handbook (version 3.0) for more information. The resuits for the first-grade sampie are not inciuded in this tabie because they did not meet WWC group 
design standards. The sampie of first-grade students experienced high attrition, and the authors were unabie to demonstrate baseiine equivaience for the anaiytic sampie. 
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Endnotes 

■' The descriptive information for this program was obtained from a publicly available source: the program’s website (http://www.renaissance. 
com, downloaded November 2014). The WWC requests developers review the program description sections for accuracy from their 
perspective. The program description was provided to the developer In November 2014, and the WWC Incorporated feedback from the 
developer. Further verification of the accuracy of the descriptive information for this program is beyond the scope of this review. 

^ The literature search reflects documents publicly available by August 2015. The previous WWC Intervention report for this interven- 
tion was released in October 2008 and was prepared using the WWC Procedures and Standards Handbook (version 1 .0), and the 
Beginning Reading review protocol (version 1 .0). The studies in this updated intervention report were reviewed using the Standards 
from the WWC Procedures and Standards Handbook (version 3.0), along with those described in the Beginning Reading review pro- 
tocol (version 3.0). This report has also been updated to Include 25 studies that have been conducted since 2008. Of the additional 
studies, 20 were not within the scope of the review protocol for Beginning Reading, four were within the review’s scope but did not 
meet WWC group design standards, and one study (Shannon, Styers, & Siceloff, 2010) met WWC group design standards without 
reservations. A complete list and disposition of all studies reviewed are provided in the references. The report also Includes reviews of 
all previous studies that met WWC group design standards with or without reservations and resulted in a revised disposition for Ross, 
Nunnery, and Goldfeder (2004). The study, which met WWC design standards without reservations in the previous report, does not 
meet WWC design standards in this report. The revised disposition is due to changes in the WWC’s attrition standards since publica- 
tion of the previous report. For Ross, Nunnery, and Goldfeder (2004), the randomized controlled trial included outcomes that had a 
combination of overall and differential attrition rates that exceeded the WWC standards (version 3.0), and the analytic intervention and 
comparison groups were not shown to be equivalent. The evidence presented In this report Is based on available research. Findings 
and conclusions may change as new research becomes available. 

® For criteria used In the determination of the rating of effectiveness and extent of evidence, see the WWC Rating Criteria on p. 20. These 
improvement index numbers show the average and range of individual-level Improvement indices for all findings across the studies. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2016, June). 

Beginning Reading intervention report: Acceierated Reader™. Retrieved from http://whatworks.ed.gov 
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WWC Rating Criteria 

Criteria used to determine the rating of a study 

study rating 

Criteria 

Meets WWC group design 
standards without reservations 

A study that provides strong evidence for an intervention’s effectiveness, such as a weii-implemented RCT. 

Meets WWC group design 

A study that provides weaker evidence for an intervention's effectiveness, such as a QED or an RCT with high 

standards with reservations 

attrition that has established equivaience of the analytic samples. 

Criteria used to determine the rating of effectiveness for an intervention 

Rating of effectiveness 

Criteria 

Positive effects 

Two or more studies show statisticaiiy significant positive effects, at least one of which met WWC group design 
standards for a strong design, AND 

No studies show statistically significant or substantively important negative effects. 

Potentially positive effects 

At least one study shows a statistically significant or substantively important positive effect, AND 

No studies show a statistically significant or substantively important negative effect AND fewer or the same number 
of studies show indeterminate effects than show statistically significant or substantively important positive effects. 

Mixed effects 

At least one study shows a statistically significant or substantively important positive effect AND at least one study 
shows a statistically significant or substantively important negative effect, but no more such studies than the number 
showing a statistically significant or substantively important positive effect, OR 

At least one study shows a statistically significant or substantively important effect AND more studies show an 
indeterminate effect than show a statistically significant or substantively important effect. 

Potentially negative effects 

One study shows a statistically significant or substantively important negative effect and no studies show 
a statistically significant or substantively important positive effect, OR 

Two or more studies show statistically significant or substantively important negative effects, at least one study 
shows a statistically significant or substantively important positive effect, and more studies show statistically 
significant or substantively important negative effects than show statistically significant or substantively important 
positive effects. 

Negative effects 

Two or more studies show statistically significant negative effects, at least one of which met WWC group design 
standards for a strong design, AND 

No studies show statistically significant or substantively important positive effects. 

No discernible effects 

None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Criteria used to determine the extent of evidence for an intervention 

Extent of evidence 

Criteria 

Medium to large 

The domain includes more than one study, AND 

The domain includes more than one school, AND 

The domain findings are based on a total sample size of at least 350 students, OR, assuming 25 students in a class, 
a total of at least 14 classrooms across studies. 

Small 

The domain includes only one study, OR 

The domain includes only one school, OR 

The domain findings are based on a total sample size of fewer than 350 students, AND, assuming 25 students 
in a class, a total of fewer than 14 classrooms across studies. 
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Glossary of Terms 

Attrition 

Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Intervention 
Intervention report 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 


Single-case design 


Attrition occurs when an outcome variable is not avaiiabie for aii participants initiaiiy assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review and inclusion in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent 
of evidence levels are given in the WWC Rating Criteria on p. 20. 

Along a percentile distribution of individuals, the improvement index represents the gain 
or loss of the average individual due to the intervention. As the average individual starts at 
the 50th percentile, the measure ranges from -50 to +50. 

An educational program, product, practice, or policy aimed at improving student outcomes. 

A summary of the findings of the highest-quality research on a given program, product, 
practice, or policy in education. The WWC searches for all research studies on an interven- 
tion, reviews each against design standards, and summarizes the findings of those that 
meet WWC design standards. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which study participants are 
assigned to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which eligible study participants are 
randomly assigned to intervention and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the 
research design and the magnitude, statistical significance, and consistency in findings. The 
criteria for the ratings of effectiveness are given in the WWC Rating Criteria on p. 20. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 


Accelerated Reader™ Updated June 2016 


Page 21 


WWC Intervention Report 


Glossary of Terms 


Standard deviation The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample tend to be spread out over a large range of values. 

Statistical significanco statistical significance is the probability that the difference between groups is a result of 

chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < .05). 


SubStantiVGly important a substantively important finding Is one that has an effect size of 0.25 or greater, regardless 

of statistical significance. 

Systematic rovlow a review of existing literature on a topic that is identified and reviewed using explicit meth- 
ods. A WWC systematic review has five steps: 1) developing a review protocol; 2) searching 
the literature; 3) reviewing studies, including screening studies for eligibility, reviewing the 
methodological quality of each study, and reporting on high quality studies and their find- 
ings; 4) combining findings within and across studies; and, 5) summarizing the review. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 
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Intervention 

Report 



Practice 

Guide 



Quick 

Review 


Single Study 
Review 



An intervention report summarizes the findings of high-quality research on a given program, practice, or policy in 
education. The WWC searches for ail research studies on an intervention, reviews each against evidence standards, 
and summarizes the findings of those that meet standards. 


This intervention report was prepared for the WWC by Mathematica Policy Research under contract ED-IES-1 3-C-001 0. 
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